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APPARATUS AND PROCESS FOR RE-TIMING VIDEO CUTS 



BACKGROUND OF THE INVENTION 

This invention concerns the processing of television signals or video 
data to obtain interlaced signals in a way which avoids the formation of 
incorrectly interlaced output. 

5 Television represents moving images by reproducing a sequence of 

"still 11 images, each of which represents a temporal sample of an original 
moving image. Many television systems use interlaced" scanning formats in 
which bandwidth is saved by dividing the scanning lines which make up a 
complete image into (usually two) cyclically repeating groups, and sending 
10 only one of these groups for each temporal sample. The full vertical resolution 
of the image is thus not achieved until a number of temporal samples have 
been displayed; and, the full temporal resolution is only available at a lower 
vertical resolution. 

The widely-used 525- and 625-line television systems use an interlace 
1 5 factor of two so that two fields of half the total number of lines make up a 
frame (alternatively known as a picture). 

More recently a number of television formats (especially high-definition 
formats) have been designed which do not use interlace; the full number of 
image lines is provided at each temporal sample. However, some 
20 transmission systems for these formats divide the image into two fields to 
improve compatibility with equipment designed for interlaced formats. 

The relevant system standards define the order in which the two fields 
of a frame are transmitted. Unfortunately not all systems use the same order; 
sometimes the topmost line of the picture is in the first field, sometimes it is in 
25 the second. This choice is known as field dominance, and there are a number 
of situations in which it is critical that it is correctly observed. 

Modern image processing techniques, compression for example, may 
combine information from more than one line of the picture, and it is not 
unusual to combine lines from different fields of the same frame. These 
30 processes will fail, or be degraded, if the field dominance is wrong. 



There are several sources of field dominance errors. The simplest is 
where a cut from one image sequence to another occurs between the two 
fields of the same frame; in tills case only half the lines will be available for the 
last image of the first sequence and for the first image of the second 
5 sequence. More complex problems can be caused when the temporal 

sampling rate of the images differs from that of the transmission system, for 
example where the speed of action is deliberately changed for effect 

For these reasons it is widely recognised that a method of 
correcting field dominance errors in a television signal is required. Known 
1 0 methods involve manually editing a recorded version of the programme. 
Usually cuts are located and incorrectly interlaced frames are removed. 
Alternatively a defective frame may be replaced by a repeat of a complete 
existing frame. Obviously deletion of frames changes the length of the 
programme, and may necessitate changes to the accompanying soundtrack. 

1 5 These manual repair techniques are very time consuming and an 

automated process, especially one combined with a format conversion 
process offers many advantages. 

Several methods of automatically detecting cuts are known (for 
example US Patent 6,075,818) and these methods may be utilised to prevent 
20 combining information across a cut in processes such as standards 

conversion, composite decoding, compression, image manipulation (e.g. 
warping) and image stabilisation. Noise reduction processes can be improved 
by using knowledge of cuts as disclosed in US Patent 6,259,489. 



25 SUMMARY OF THE INVENTION 

There can be provided a video process having an interlaced sequence 
of output fields comprising an output frame sequence wherein shot changes 
are re-timed automatically to coincide with frame boundaries and the said re- 
timing does not affect the number of output fields. 

30 The invention also comprises a method of improving the presentation 

of shot changes in a video sequence of interlaced fields by automatically 
replacing at least one field at a shot change with a field derived from a 

-2- 



preceding or succeeding field. Suitably, the replacement field is derived by 
copying the lines of a preceding or succeeding field. Alternatively, the 
replacement field is derived by spatial interpolation of a preceding or 
succeeding field. Advantageously, the replacement field is derived by motion 
compensated processing. 

In a further aspect the invention comprises a method of creating a 
sequence of interlaced video fields from a video signal or video data wherein 
picture information is automatically shifted in time at a shot change so that 
field dominance is respected and the said time shifting does not alter the 
duration of the sequence. 

In yet another aspect the invention consists of a method of 
automatically processing video or video data in response to associated shot- 
change data to give an interlaced output wherein the constituent fields of each 
processed frame are all derived from the same shot. 

15 In one example, the present invention consists in a video process 

comprising the steps of receiving an interlaced sequence of input fields 
organised in a plurality of frames; identifying a cut between first and second 
input fields; identifying whether the cut occurs at a frame boundary and, where 
a cut occurs otherwise than at a frame boundary, generating from said second 

20 field a synthetic field and replacing said first field by said synthetic field, the 
process thereby outputting a interlaced sequence of output fields in which the 
cut is positioned at a frame boundary, the sequence of output fields containing 
the same number of fields as the sequence of input fields. 

In another example, the present invention consists in video 
25 processing apparatus comprising a video input adapted to receive an 
interlaced sequence of input pictures organised in a plurality of frames; a 
control input adapted to receive video cut information; means for identifying a 
video cut occurring otherwise than at a frame boundary; and processing 
means for outputting an interlaced sequence of output pictures organised in a 
30 plurality of frames with each cut occurring otherwise than at a frame boundary 
in the input sequence being automatically retimed to occur at a frame 
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boundary in the output sequence, the sequence of output fields containing the 
same number of fields as the sequence of input fields. 

In still another example, the present invention consists in a video 
process comprising the steps of receiving a sequence of input fields organised 
5 in a plurality of frames; identifying a cut between first and second input fields; 
identifying whether the cut occurs at a frame boundary and, where a cut 
occurs otherwise than at a frame boundary, retiming the cut, the process 
thereby outputting a interlaced sequence of output fields in which each cut is 
positioned at a frame boundary, the sequence of output fields containing the 
1 0 same number of fields as the sequence of input fields. 

In yet another example, the present invention consists in video 
processing apparatus comprising a video input adapted to receive a sequence 
of input frames; a video output adapted to provide a sequence of output fields 
organised in a plurality of frames; a field predictor adapted to receive a base 
1 5 field and to generate therefrom a synthetic field having a different timing; a 
field sequence detector; a video cut detector; and a field substitution element 
controlled through said field sequence detector and said video cut detector to 
substitute a synthetic field at a cut occurring otherwise than at a frame 
boundary, thereby to retime the cut to occur at a frame boundary in the output 
* 20 sequence. 

BRIEF DESCRIPTION OF DRAWINGS 

An example of the invention will now be described with reference to the 
drawings in which: 

25 Figure 1 shows a video process with field dominance correction. 

Figure 2a shows a sequence of fields input to the dominance processor 
of the invention. 

30 Figure 2b to 2e inclusive show sequences of output fields in 

accordance with various modes of operation of the invention. 
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Figure 21 shows an alternative sequence of fields input to the 
dominance processor of the invention. 

5 Rgure 3 shows a block diagram of an embodiment of the invention. 

Figure 4 shows a block diagram of an alternative embodiment of the 
invention. 

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS 

10 Referring to Figure 1, an Input video sequence (1), which may In any 

format analogue, digital or file-structured digital; and may have an interlaced 
or progressive sampling structure, is input to a video process (2). The video 
has a temporal sampling structure such that different groups of spatial 
samples correspond to different moments in time. If the video is interlaced 

1 5 these groups will be fields, and if the video is progressive these groups will be 
frames. 

Associated with the video (1 ) is cut or shot change data (3). This data 
has a value for each temporal sampling point of the video (1) indicating 
whether the image represented at that sampling time is the same scene as 

20 was represented at the immediately preceding sampling time, or whether the 
samples correspond to a new Image of a different scene. The data (3) may be 
derived from the video (1 ) by a shot-change detector (4) r or it may already be 
encoded into the video (1) or be In the form of metadata (such as an edit 
decision list, EDL) associated with the video. The shot-change detector (4), if 

25 present, operates in known manner, for example as described in US Patent 
6,075,818. 

The video process (2) can be arbitrary, its output may be interlaced or 
progressive and may have a temporal and/or spatial sampling structure 
different from its input Preferably the process makes use of the data (3) to 
30 avoid combining samples from different scenes. The processed output video 
(5) is associated with processed shot-change data (6) indicating whether each 
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temporally coincident set of output video samples represents the same scene 
as the preceding set. 

If there is no temporal re-sampling involved in the process (2) the data 
(6) may simply be a delayed version of the data (3). If there is temporal 
processing it will be necessary to track the correspondence between input and 
output temporal samples in order to construct the output data (6). 

The processed video (5) and the shot-change data (6) are input to a 
field dominance processor (7), which produces an interlaced video output (8). 
The processor (7) operates (as will be described below) to ensure that shot 
changes at the output (8) always occur at the same point in the sequence of 
interlaced fields. 

Figure 2a shows an example of three frames of a possible 2:1 
interlaced input to the field dominance processor. The horizontal axis of the 
figure represents the timing of the output fields, and the vertical axis 
represents the timing of the input fields. The first output frame comprises two 
interlaced fields (201 ) and (202), the next frame comprises the interlaced 
fields (203), (204) and so on. 

The fields (201), (202) and (203) represent successive temporal 
samples of a first scene, and the immediately following fields (204), (205) and 
(206) represent successive temporal samples of a second scene. There is, 
therefore, a cut between the two scenes at the start of the second field (204) 
of the second frame, and the two fields (203) (204) of this frame contain 
material from different scenes so that the field dominance is incorrect As 
mentioned above, a prior-art method of correcting this error is to edit out the 
frame comprising the fields (203) and (204), with consequent change to the 
running time of the video sequence. 

In one example of the invention the field dominance processor replaces 
the field (204) with a new, synthetic field constructed from the field (203). The 
result is shown in Figure 2b; it can be seen that the field dominance is now 
correct, but there is a distortion of the motion profile of the first scene 
immediately prior to the cut. 



Another example of the invention is shown in Figure 2c. Here the field 
(203) is replaced by a new field constructed from the field (204). Again the 
field dominance error is corrected but now the motion profile of the second 
scene is distorted immediately after the cut This is preferable to the method 
5 of Figure 2b because the eye is more tolerant of motion profile distortion after 
a change of scene. 

It is possible to avoid the motion profile distortion by using known 
motion compensated prediction methods to construct the new fields. 
Examples are shown in Figure 2d (equivalent to Figure 2c) and Figure 2e 
1 0 (equivalent to Figure 2b). Although the video is derived from incorrectly timed 
temporal samples, the positions of moving objects in the image are corrected 
by the use of motion vectors. 

The input to the dominance processor may not have a temporal 
sampling rate equal to the frame rate (as in Figure 2a). It is common practice 

15 to present film with a frame rate of 25 Hz by means of 50 Hz-rate interlaced 
fields; it is also common to "package" progressive frames as a pair of 
interlaced fields. These both amount to the situation shown in Figure 2f. 
Referring to this Figure it can be seen that both fields of each frame 
correspond to the same temporal sample of the original image. A field 

20 dominance error due to a cut is shown at the fields (203) (204). This error can 
be corrected in the same way as described previously by replacing the field 
(203) by information from the field (204) or vice versa. In this case there is no 
need for motion compensation as only spatial prediction is required. 
(However, if motion compensation is available it may be possible to improve 

25 the prediction by using information from other frames of the same scene.) 

Some examples of the method of operation of the field dominance 
processor of the invention will now be described. Figure 3 shows a method of 
carrying out the operation shown in Figure 2b. A 2;1 interlaced video input 
(301) has associated shot-change data (302). A dominance-corrected video 
30 output (303) is derived from a changeover switch (304) which selects either 
the input video (301 ), or the output of a video predictor (305). 



-7- 



The switch (304) is controlled in dependence upon the shot-change 
data (302) and field-sequence data (306) derived from the Input video (301). 
Normally the switch (304) selects the video (301 ) for output However, when 
the shot-change data (302) indicates that the current input video field 
5 corresponds to a new scene, and the current field is not the first of a new 
frame (as indicated by the field-sequence data (306)), the switch takes a field 
from the predictor (305) in place of the field from the input (301 ). 

The predictor (305) could use any of the known video prediction 
methods. The simplest predictor is a delay corresponding to an integral 

1 0 number of lines close to one field period. This is a very crude prediction and 
will give vertical position errors. These errors can be avoided by using an 
interpolator taking two or more lines of the input field to create the prediction. 
It would even be possible to use information from preceding frames in the 
interpolation, providing that the associated shot-change data shows that these 

15 frames are from the same scene as the one being processed. 

The best prediction would be a motion compensated prediction making 
use of motion vectors, either newly-derived by a motion estimator (307), or 
already available either as part of the video (301), or associated with it. In the 
known art of motion-compensated prediction pixels or groups of pixels are 
20 moved to different positions (for example by manipulating the addressing of a 
store) in accordance with vectors describing the motion of the objects they 
represent. In this way the predicted field can be made to have the content of 
the previous field, but with objects in the positions which they would have 
occupied at the time of the current field. 

25 If the shot-change data (302) is derived by the method of US Patent 

6 ? 075,818 (the contents of which are hereby incorporated by way of 
reference) the output (or intermediate data) from the motion estimator (307) 
can be used in its derivation. 

Figure 4 shows a method of carrying out the process shown in 
30 Figure 2c, The method is similar to that shown in Figure 3, and similar 

functions have the same identifying numerals, but with the prefix '3' replaced 
by '4\ In order to obtain ready access to the field after the current field, the 
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'normal 1 video path is via an integraMine-number delay close to one field 
period (410). The predictor (405) has a short delay so that when its output is 
selected by the switch (404), a field having the content of the field that follows 
the current output field is output As with the previous example, the prediction 
5 can be by simple substitution of lines from the succeeding field, or by a 
suitable interpolation process, which could be motion compensated. 

The invention is also applicable to the case where the input video is 
progressive and an interlaced output with correct field dominance is required. 
If progressive frames are input, and the frame rate of the input is similar to the 
10 field rate of the output, output fields may be easily constructed by spatially 
sub-sampling the input frames. However, as shot changes can occur on any 
input frame, there will be no guarantee that the output field dominance will be 
correct. 

This problem may be solved by the invention as follows. Either the 
1 5 structure of Figure 3 or Figure 4 may be used, provided that the processing is 
modified so as always to include the necessary vertical sub-sampling in the 
video path. The prediction function is greatly simplified because any set of 
interlaced lines can be taken from any input frame; however, motion 
compensation of the prediction is still necessary to avoid motion profile 
20 distortion at cuts. 
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The invention has been described by way of example and other 
embodiments are possible. For example the technique can be applied to 
systems having more than two fields in each frame; in this case more than 
one field will need to be replaced by a predicted field at a wrongly-timed cut. 
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